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Abstract — We describe a family of instanton-based optimiza- 
tion methods developed recently for the analysis of the error 
floors of low-density parity-check (LDPC) codes. Instantons are 
the most probable configurations of the channel noise which 
result in decoding failures. We show that the general idea and 
the respective optimization technique are applicable broadly to 
a variety of channels, discrete or continuous, and variety of 
sub-optimal decoders. Specifically, we consider: iterative belief 
propagation (BP) decoders, Gallager type decoders, and linear 
programming (LP) decoders performing over the additive white 
Gaussian noise channel (AWGNC) and the binary symmetric 
channel (BSC). 

The instanton analysis suggests that the underlying topological 
structures of the most probable instanton of the same code 
but different channels and decoders are related to each other. 
Armed with this understanding of the graphical structure of the 
instanton and its relation to the decoding failures, we suggest a 
method to construct codes whose Tanner graphs are free of these 
structures, and thus have less significant error floors. 

Index Terms — Low-density parity-check codes. Error Floor, 
Iterative Decoding, Linear Programming Decoding, Instantons, 
Pseudo-Codewords, Trapping Sets 



I. Introduction 

LDPC codes H], 12], have been the focus of intense research 
over the past decade because they can approach theoretical 
limits of reliable transmission over various channels even when 
decoded by sub-optimal low complexity algorithms. 

Two important classes of such algorithms are (i) iterative de- 
coding algorithms, which include message passing algorithms 
(variants of the BP algorithm Q and Gallager type algorithms 
IT]), and bit flipping algorithms ||4], Q (serial and parallel), 
as well as (ii) the LP decoding algorithm ||6] . Characterization 
of the error performance of sub-optimal algorithms (or simply 
decoders) is still an open problem, and has been addressed for 
both LDPC code ensembles, as well as for individual codes 
17] . Error performance of LDPC codes in the asymptotic limit 
of the code length is well characterized for a large class of 
sub-optimal decoders over different channels (the interested 
reader is referred to H], Is], 15], ifTol for general theory of 
message passing algorithms, l4], ||5], ifTTl . fl2i for analysis 

Manuscript received October 1, 2008. Revised January 15, 2009 and Mai'ch 
6, 2009. S. K. Chilappagari [shashic@ece.arizona.edu] is with the Electrical 
and Computer Engineering Department, University of Arizona, Tucson, AZ, 
85721, USA. 

M. Chertkov [chertkov@lanl.gov] is with Theory Division & CNLS, LANL, 
Los Alamos, NM, 87545 USA. 

M. G. Stepanov [stepanov@math.arizona.edu] is with the Department of 
Mathematics, University of Arizona, Tucson, AZ, 85721, USA. 

B. Vasic [vasic@ece.arizona.edu] is with the Electrical and Computer 
Engineering Department, University of Arizona, Tucson, AZ, 85721, USA. 



of bit flipping algorithms and expander based arguments and 
O, El, ES] for analysis of the LP decoder). 

A common feature of all the analysis methods used in 
deriving the asymptotic results is that the underlying as- 
sumptions hold in the limit of infinitely long code and/or 
are applicable to an ensemble of codes. Hence, they are of 
limited use for the analysis of a given finite length code. The 
performance of a code under a particular decoding algorithm 
is characterized by the bit-error-rate (BER) or the frame- 
error-rate (FER) curve plotted as a function of the signal-to- 
noise ratio (SNR). A typical BER/FER vs SNR curve consists 
of two distinct regions. At small SNR, the error probability 
decreases rapidly with SNR, with the curve looking like a 
waterfall. The decrease slows down at moderate values turning 
into the error floor asymptotic at very large SNR [|T6l . This 
transient behavior and the error floor asymptotic originate 
from the sub-optimality of decoder, i.e., the ideal maximum- 
likelihood (ML) curve would not show such a dramatic change 
in the BER/FER with the SNR increase. While the slope of 
the BER/FER curve in the waterfall region is the same for 
almost all the codes in the ensemble, there can be a huge 
variation in the slopes for different codes in the error floor 
region Q. Since for sufficiently long codes the error floor 
phenomenon manifests itself in the domain unreachable by 
brute force Monte-Carlo (MC) simulations, analytical methods 
are necessary to characterize the FER performance. 

Finite length analysis of LDPC codes is well understood 
for decoding over the binary erasure channel (BEC). The 
decoder failures in the error floor domain are governed by 
combinatorial structures known as stopping sets IT?]. Stopping 
set distributions of various LDPC ensembles have been studied 
by Orlitsky et al. (see llTSl and references therein for related 
works). Unfortunately, such a level of understanding of the 
decoding failures has not been achieved for other important 
channels such as the BSC and the AWGNC. 

In this paper, we focus on the decoding failures of LDPC 
codes for iterative as well as LP decoders over the BSC and 
the AWGNC. Failures of iterative decoders for graph based 
codes were first studied by Wiberg lfT9l who introduced the 
notions of computation trees and pseudo-codewords. Subse- 
quent analysis of the computation trees was carried out by 
Frey et al. 1201 and Forney et a/. 1211 . The failures of the 
LP decoder can be understood in terms of the vertices of 
the so-called fundamental polytope which are also known as 
pseudo-codewords (6] . Vontobel and Koetter ll22l introduced a 
theoretical tool known as graph cover approach and used it to 
establish connections between the LP and the message passing 



decoders using the notion of the fundamental polytope. They 
showed that the pseudo-codewords arising from the Tanner 
graph covers are identical to the pseudo-codewords of the LP 
decoder. Vontobel and Koetter 1231 also studied the relation 
between the LP and the min-sum decoders. 

For iterative decoding on the AWGNC, MacKay and Postol 
ll24l were the first to discover that certain "near codewords" 
are to be blamed for the high error floor in the Margulis 
code. Richardson |161 reproduced their results and developed a 
computation technique to predict the performance of a given 
LDPC code in the error floor domain. He characterized the 
troublesome noise configurations leading to the error floor 
using combinatorial objects termed trapping sets and described 
a technique (of a Monte-Carlo importance sampling type) to 
evaluate the error rate associated with a particular class of 
trapping sets. The method from llT6l was further refined for 
the AWGNC by Stepanov et al. ||25l who introduced the notion 
of instantons. In a nutshell, an instanton is a configuration of 
the noise which is positioned in between a codeword (say 
zero codeword) and another pseudo-codeword (which is not 
necessarily a codeword). Incremental shift (allowed by the 
channel) from this configuration toward the zero codeword 
leads to correct decoding (into the zero-codeword) while 
incremental shift in an opposite direction leads to a failure. 
In principle, one can find this dangerous configuration of the 
noise by exploring the domain of correct decoding surrounding 
the zero codeword, and finding borders of this domain - the 
so-called error- surface. If the channel is continuous, the error- 
surface consists of continuous patches while configuration of 
the noise maximizing the error probability over a patch is 
called an instanton. The term instanton introduced initially in 
the context of disordered systems is also known under the 
names of saddle-point or optimal fluctuation, and is common 
in modern theoretical physics (see 1251 and references therein). 

As stated above, the instantons that affect the decoder 
performance in the error floor region are extremely rare, and 
hence identifying and enumerating them is a challenging task. 
However, once this difficulty is overcome, the knowledge of 
the trapping set/pseudo-codeword distribution can be used to 
evaluate the performance of the code. It can also be used 
to guide optimization of the code and design of improved 
decoding strategies. In this paper, we focus on the methods 
used to identify the most relevant noise configurations for 
various decoders and channel models. 

Previous investigation of the problem include the work 
by Kelley and Sridhara l26l who studied pseudo-codewords 
arising from graph covers and derived bounds on the minimum 
pseudo-codeword weight in terms of the girth and the mini- 
mum left-degree of the underlying Tanner graph. The bounds 
were further investigated by Xia and Fu l27l . Smarandache 
and Vontobel l28ll found pseudo-codeword distributions for 
the special cases of codes from Euclidean and projective 
planes. Pseudo-codeword analysis has also been extended to 
the convolutional LDPC codes by Smarandache et al. l29l . 
Milenkovic et al. l30l studied the asymptotic distribution of 
trapping sets in regular and irregular ensembles. Wang et al. 
I3TI proposed an algorithm to exhaustively enumerate certain 
trapping sets. 



Chernyak et al. ||32ll and Stepanov et al. ||25l suggested 
to pose this problem of finding the instantons as a special 
optimization problem. This optimization method was built in 
the spirit of the general methodology, borrowed from statistical 
physics, guiding exploration of rare events which contribute 
the most to the BER/FER. The optimization method allowed 
to discover in l25l . the set of most probable instantons for 
the AWGNC and iterative decoder The operational utility of 
the method was illustrated on some number of moderate size 
examples and dependence of the instanton structure on the 
number of iterations was observed. The general optimization 
method was substantially improved and refined in l33l for 
the LP decoder over continuous channels (with main enabling 
example chosen to be the AWGNC). The pseudo-codeword- 
search (PCS) algorithm of f33l was essentially exploring in 
an iterative way the Wiberg formula, treating an instanton 
configuration as a median between a pseudo-codeword and 
the zero-codeword. 

It was shown empirically that, initiated with a sufficiently 
noisy configuration, the algorithm converges to an instanton 
in sufficiently small number of steps, independent or weakly 
dependent on the code size. Repeated multiple times, the 
method outputs the set of instanton configurations which can 
further be used to estimate the BER/FER performance in 
the transient and error floor domain. The definition of the 
instantons and the instanton search method were extended in 
l34l to the BSC. In this special case, the instanton search 
algorithm is provably efficient, in the sense that it outputs 
an instanton in small number of steps, and that the weight 
of the pseudo-codeword found in the intermediate steps is 
monotonically decreasing. (See also f35l for an exhaustive list 
of references for this and related subjects.) 

In this paper, we discuss failures of iterative decoders 
(specifically the BP algorithm and the Gallager A/B al- 
gorithms) as well as LP decoding over the BSC and the 
AWGNC. We explain the notion of instanton and elaborate 
on the connections between instantons and trapping sets as 
well as pseudo-codewords. We then describe algorithms to 
search for instantons. By using the [155, 64, 20] Tanner code 
l36l as an enabling example, we illustrate the performance 
of the instanton search technique outputting the set of most 
probable instantons. By identifying that all decoding failures 
can be attributed to the presence of certain subgraphs, we 
construct a code avoiding this subgraph and show that this 
code outperforms the original code. Throughout the paper, we 
focus on the BSC and the AWGNC and while the underlying 
approach is similar for both channels, rigorous statements can 
be made for the BSC l34|, while the respective AWGNC 
statements come from experiments only. 

The rest of the paper is organized as follows. In Section HIl 
we introduce the notation and provide the required background 
material. The notions of decoding failures and instantons are 
discussed in Section HIH followed by a description of instanton 
search algorithms for different decoders in Section |IV] We 
illustrate numerical results in Section [V] and conclude in 
Section rvll 



II. Preliminaries 



C. Decoding Algorithms 



A. LDPC Codes 

LDPC codes belong to the class of linear block codes which 
can be defined by sparse bipartite graphs ll37l . The Tanner 
graph |[3?1 G of an LDPC code C is a bipartite graph with 
two sets of nodes: the set of variable nodes V = {1, 2, . . . , n} 
and the set of check nodes C = {1,2,..., m}. The check 
nodes (variable nodes resp.) connected to a variable node 
(check node resp.) are referred to as its neighbors. The set 
of neighbors of a node u is denoted by M{u). The degree 
du of a node u is the number of its neighbors. A vector 
V = (wi, 7J2, . . . , fn) is a codeword if and only if for each 
check node, the modulo two sum of its neighbors is zero. 
An {n,dy,dc) regular LDPC code has a Tanner graph with n 
variable nodes each of degree dy and nd^/dc check nodes each 
of degree dc- This code has length n and rate r > l—dy/dc Q. 
It should be noted that the Tanner graph is not uniquely defined 
by the code and when we say the Tanner graph of an LDPC 
code, we only mean one possible graphical representation. 



B. Channel Assumptions 

We assume that a binary codeword y is transmitted over a 
noisy channel and is received as y. The support of a vector 
y = (j/i, ?/2, • ■ • , Vn), denoted by supp(y), is defined as the set 
of all positions i such that yi ^ 0. In this paper, we consider 
binary input memoryless channels with discrete or continuous 
output alphabet. As the channel is memoryless, we have 



Pr(y|y) ^Xl'PAUyt) 



and hence can be characterized by Pi{yi\yi), the probability 
that yi is received given that y^ was sent. The negative log- 
likelihood ratio (LLR) corresponding to the variable node i E 
V is given by 



It = log 



Pr{y,\y, =. 0) 
Pr(y.|y. - 1) 



Two binary input memoryless channels of interest are the 
BSC with output alphabet {0, 1} and the AWGNC with output 
alphabet R. On the BSC with transition probability e, every 
transmitted bit yi e {0, 1} is flipped Q with probability e and 
is received as yi £ {0,1}. Hence, we have 




For the AWGNC, we assume that each bit yi e {0, 1} 
is modulated using binary phase shift keying (BPSK) and 
transmitted as = 1 — 2yi and is received as yi = yi + Ui, 
where {n^} are i.i.d. iV(0, cr^) random variables. Hence, we 
have 

2y^ 

jt = —■ 

'The event of a bit changing from to 1 and vice- versa is known as 
flipping. 



1) Message Passing Decoders: Message passing decoders 
operate by passing messages along the edges of the Tanner 
graph representation of the code. Gallager in HI proposed 
two simple binary message passing algorithms for decoding 
over the BSC; Gallager A and Gallager B. There exist a large 
number of message passing algorithms (the BP algorithm, the 
min-sum algorithm, quantized decoding algorithms, decoders 
with erasures [81 to name a few ) in which the messages belong 
to a larger alphabet. 

Let y = {yi,y2, ■ ■ ■ ,yn), an n-tuple be the input to the 

(k) 

decoder. Let ujIJ,^ denote the message passed by a variable 
node i £ V to its neighboring check node a e C in the /c*'' 

(k) 

iteration and w^^^ denote the message passed by a check 
node a to its neighboring variable node i. Additionally, let 
'^i'^i denote the set of all incoming messages to variable 



(k) 

node i and zu\ denote the set of all incoming messages 



to variable node i except from check node a. 
Gallager A/B Algorithm: The Gallager A/B algorithms are 
hard-decision-decoding algorithms in which all the messages 
are binary. With a slight abuse of the notation, let |-ni7»_,i = 
m| denote the number of incoming messages to i which are 
equal to m G {0, 1}. Associated with every decoding round 
k and variable degree d, is a threshold bk^di- The Gallager B 
algorithm is defined as follows. 



,(0) 



w 



(k) 



,(k- 



UeA/'(a)\i 



,(fc) 



if = 1| > b,.d^ 

if = o| > bkM. 



1, 

yi, otherwise 



The Gallager A algorithm is a special case of the Gallager B 
algorithm with bk,di = di — 1 for all k. At the end of each 
iteration, a decision on the value of each variable node is made 
based on all the incoming messages and possibly the received 
value. 

The BP Algorithm: A soft-decision-decoding algorithm, 
which is the best possible one if the messages are calculated 
locally in the Tanner graph of the code, is the BP algorithm 
(also known as the sum-product algorithm). With a moderate 
abuse of notation, the messages passed in the BP algorithm 
are described below: 



,(0) 



,{k) 



It 

2 tanh" 



n tanh(i 

\jeAA(a)\» ^ 



It 



E 



(56A^(i)\a 



The result of decoding after k iterations, denoted by x 
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determined by the sign of m^*^' = 
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0, otherwise a; 
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In the limit of high SNR, when the absolute value of the 
messages is large, the BP algorithm becomes the min-sum 
algorithm, where the message from the check a to the bit i 
looks like: 

(fc) . I (fc-l) I T\ I 

jeA/'(Q)\i 

The min-sum algorithm has a property that the Gallager 
A/B and the LP decoders also possess — if we multiply all 
the Ukelihoods 7^ by a factor, all the decoding would proceed 
as before and would produce the same result. Note that we do 
not have this "scaling" in the BP algorithm. 

To decode the message in complicated cases (when the 
message distortion is large) we may need a large number 
of iterations, although typically a few iterations would be 
sufficient. To speed up the decoding process one may check 
after each iteration whether the output of the decoder is a valid 
codeword, and if so halt decoding. 

2) Linear Programming Decoder: The ML decoding of 
the code C allows a convenient LP formulation in terms of 
the codeword polytope poly(C) i6j whose vertices correspond 
to the codewords in C. The ML-LP decoder finds f = 
(/i, . . . , /„) minimizing the cost function ^11=1 lifi subject 
to the f € poly(C) constraint. The formulation is compact but 
impractical as the number of constraints is exponential in the 
code length. 

Hence a relaxed polytope is defined as the intersection of 
all the poly topes associated with the local codes introduced for 
all the checks of the original code. Associating (/i, . . . , /„) 
with bits of the code we require 

</.<!, -ii^v (1) 

For every check node a, let M{a) denote the set of variable 
nodes which are neighbors of a. Let Ea = {T Q JV{a) : 
\T\ is even}. The polytope Qa associated with the check node 
a is defined as the set of points (f , w) for which the following 
constraints hold 

< Wa,T < 1, VT e (2) 

fi = J2TeE^,T3i Wa,T, Vi € U [o) (4) 

Now, let Q — DaQa be the set of points (f , w) such that 
(|4]i hold for all a G C. (Note that Q, which is also referred 
to as the fundamental polytope ll38l . 1221 . is a function of 
the Tanner graph G and consequently the parity-check matrix 
H representing the code C.) The linear code linear program 
(LCLP) can be stated as 

min ^^7^/^, s.t. (f,w) E Q. 

For the sake of brevity, the decoder based on the LCLP is 
referred to in the following as the LP decoder. A solution 
(f , w) to the LCLP such that all fjS and w^.ts are integers is 
known as an integer solution. The integer solution represents 
a codeword [|6]|. It was also shown in 16j that the LP decoder 
has the ML certificate, i.e., if the output of the decoder is a 
codeword, then the ML decoder would decode into the same 



codeword. The LCLP can fail, generating an output which is 
not a codeword. 

It is appropriate to mention here that the LCLP can be 
viewed as the zero temperature version of BP-decoder looking 
for the global minimum of the so-called Bethe free energy 
functional ll39l . 

III. Decoding Failures and Instantons 

To characterize the performance of a coding/decoding 
scheme for linear codes over any output symmetric channel, 
one can assume, without loss of generality, the transmission 
of the all-zero-codeword, i.e. y = 0, when the decoding 
algorithm satisfies certain symmetry conditions (see Definition 
1 and Lemma 1 in |8||). The iterative decoding algorithms that 
we consider in this paper satisfy these symmetry conditions. 
The assumption of the transmission of the all-zero-codeword 
also holds for the LP decoding of linear codes on output 
symmetric channels, as the polytope Q is highly symmetric 
and looks exactly the same from any codeword (see Q for 
proof). Henceforth, we assume that y = 0. 

A decoding failure is said to have occurred if the output of 
the decoder is not equal to the transmitted codeword (all-zero- 
codeword). Probability of a decoder failure, or the frame error 
rate as a function of the SNR s{= Eb/No) can be expressed 
as: 

FER{s)=Y,Ps{v)0{y), (5) 
y 

where the sum goes over all the possible outputs of the 
channel for the zero-codeword input. In case of a continuous 
output channel, the sum becomes an integral: ^ / '^V^ ^^'^ 
the channel probability mass function becomes a probability 
density function: j dyPs{y) = 1. 0{y) in Eq. (|5]i is defined 
to be zero, in the case of successful decoding, and is unity in 
the case of failure. Ps{y) is the probability of observing y at 
the output of a channel characterized by the SNR s 0. 

Calculating the above sum/integral exactly is not feasible, 
and the instanton-based approach consists of approximating 
the sum/integral by a finite number of terms corresponding to 
the most probable failures - the instantons. This approximation 
becomes asymptotically exact in the limit of large SNR, 
while at smaller SNRs, more terms are needed to obtain 
accurate approximation for the FER. Note that the details 
of the approximate evaluations are different for discrete and 
continuous channels. In the discrete case, the number of terms 
is finite. We account for the fc-most probable configurations, 
and FER{s) « 'T%=i-^0Ps{yi3), where the multiplicity 
factor Mp counts the number of instantons equivalent under 
bit permutations. For continuous channels, an instanton is 
a stationary point of the respective integrand. By stationary 
point, we mean the local maximum of the noise probability 
density function. Hence for the AWGNC, instanton is defined 
as the noise configuration with minimal (probably locally) 
value of the norm of y that leads to a decoding failure. 

^Note that for the BSC, the transition probability e is a measure of the 
SNR. For code rate r and BPSK modulated transmission over the AWGNC 
with noise variance a^, we have Ei,/No = l/{2ra-^). 




Fig. 1. Illustration of error surface. 



The norm of a vector y is equal to y X^iev Vi^ ■ Note that 
for the AWGNC, smaller the norm, the more probable the 
noise configuration is. 

The FER approximation should also include, in addition 
to the multipUcities, the curvature corrections around the 
stationary point (e.g. within Gaussian approximation) 1321 . 
EQ). In other words, FER{s) « T.'}=i^pCs{yp)Ps{yp), 
where Cs{yp) is the curvature factor. The multiplicities of the 
instantons are determined by the symmetry group of the code, 
and if nothing is known about it, one may assume that the 
group is trivial and all multiplicities are equal to 1. In contin- 
uous case, the curvatures are determined by the geometry of 
the error surface in the vicinity of the instanton (this subject 
was not studied numerically so far, as the most important 
information about the instanton is its weight which determines 
the slope of FER vs. SNR curve in the asymptotic). Intuitively, 
in the case of the AWGNC and s -> oo, Cs{yfj) = 0{l/^/s), 
the decay of the noise correlations is exponential along one 
direction (orthogonal to the error surface) and quadratic along 
the remaining — 1 components of the noise vector (see Fig. 
[T]for an illustration of the error surface). 

Consistent with the above statements, instantons can be 
also defined as special configurations of the noise resulting 
in decoding failures such that any incremental (and channel 
specific) shift of the noise toward the zero-codeword results in 
correct decoding. It is thus useful to also introduce a respective 
output, = dec(ys), called a pseudo-codeword. It should be 
noted that this informal definition of the pseudo-codewords 
is generic and applicable to any channel and decoder While 
the output for the LP decoder is well defined and does not 
suffer from numerical issues, the iterative decoder can exhibit 
oscillations i.e., the bits which are decoded wrongly can differ 
from one iteration to another. As a way to streamline the 
description of decoding failures in the presence of rounding 
and iterative uncertainties, Richardson lfT6l suggested a proxy 
notion of the trapping set, which is a combinatorial object that 
accounts for the decoder output over iterations. In the subse- 
quent discussion, we formally define trapping sets and pseudo- 
codewords and also provide some BSC-specific definitions. If 
an instanton of a channel/decoder is known, the respective 
pseudo-codeword can be easily found, and conversely if a 
pseudo-codeword is given (i.e. we know for sure that there 
exists a configuration of the noise which is sandwiched in 
between the pseudo-codeword and the zero-codeword) the 



respective instanton can be restored. In fact, this inversion is in 
the core of the pseudo-codeword/instanton search algorithms 
discussed in Section HVl 

A. Trapping Sets for Iterative Decoders 

In practice, we assume that the iterative decoder performs 
a finite number D of iterations. Let y = (yi,y2, ■ ■ ■ ,yn) be 
a vector which is the input to the iterative decoder and let 
x^*^^ = (xj'^-' , ^2*^ xl'^''), fc < I? be the output binary 
vector at the fc*'* iteration. A variable node i is said to be 
eventually correct if there exists a positive integer K such 
that for all k > K, xf^ = |[T6]| . Formally, a decoder failure 
is said to have occurred if there does not exist k such that 
supp(x('^)) = ||T6|. 

Definition 1: 1161 (Trapping sets for iterative decoders:) For 
an input y, let T(y) denote the set of variable nodes that are 
not eventually correct. If T(y) ^ 0, then T(y) is a trapping 
set. If a = |T(y)| and b is the number of odd degree check 
nodes in the sub-graph induced by T(y), we say T(y) is an 
(a, h) trapping set. 

For the BSC, since the input to the decoder as well as the 
messages passed are discrete, it is easier to define instantons 
in terms of number of bits flipped in the input to the decoder. 
The instantons with least number of flips will be the most 
dominant in the error floor region. We formalize this intuition 
below. 

Definition 2: (Critical number for Gallager A/B algorithm) 
Let T he a trapping set for the Gallager A/B algorithm 
and let y e GF(2)". Let Y(T) = {y|T(y) = T}. The 
critical number m{T) of trapping set T for the Gallager A/B 
algorithm is the minimum number of variable nodes that have 
to be initially in error for the decoder to end up in the trapping 
set T, i.e., 

m(T) = min |supp(y)|. 

The most relevant trapping set in the error floor region is 
the trapping set with the least critical number 

Definition 3: (Instanton for Gallager A/B over the BSC) 
An instanton is a binary vector i such that T(i) = T for 
some trapping set T and for any binary vector r such that 
supp(r) C supp(i), T(r) = 0. The size of an instanton is the 
cardinality of its support. 

Given a trapping set, one can consider vectors whose sup- 
port is a subset of the trapping set as input to the decoder and 
see if such vectors are instantons. While rigorous statements 
cannot be made about finding smallest size instantons, the 
above method gives instantons in most of the cases (see BTl 
for some illustrations). Intuitively, this seems reasonable as we 
do not expect inputs to the decoder which do not have errors 
in variable nodes involved in a trapping set to end up in a 
trapping set. 

B. Pseudo-codewords for LP Decoders 

In contrast to the iterative decoders, the output of the LP 
decoder is well defined in terms of pseudo-codewords. 

Definition 4: ||6l An integer pseudo-codeword is a vector 
p = (pi, . . . ,p„) of non-negative integers such that, for every 



parity check a G C, the neighborhood {pi : i E J\f{a)} is a 
sum of local codewords. 

The interested reader is referred to Section V in @ for 
more details and examples. Alternatively, one may choose to 
define a re-scaled pseudo-codeword, p = (pi, . . . ,p„) where 
< Pi < l,Vi S V, simply equal to the output of the 
LCLP. In the following, we adopt the re-scaled definition. The 
cost associated with LP decoding of a vector y to a pseudo- 
codeword p is given by 



cost(y,p) 



For an input y, the LP decoder outputs the pseudo-codeword 
p with minimum cost(y, p). Since the cost associated with 
LP decoding of y to the all-zero-codeword is zero, a decoder 
failure occurs on the input y if and only if there exists a 
pseudo-codeword p with cost(y,p) < 0. 

A given code C may have different Tanner graph repre- 
sentations and consequently potentially different fundamental 
polytopes. Hence, we refer to the pseudo-codewords as corre- 
sponding to a particular Tanner graph G of C. 

Definition 5: 11211 Let p = (pi,...,p„) be a pseudo- 
codeword distinct from the all-zero-codeword of the code C 
represented by Tanner graph G . Then, the pseudo-codeword 
weight of p is defined as follows: 

• wbsc{p) for the BSC is 

where e is the smallest number such that the sum of the 
e largest piS is at least {J^ievPi) /2- 

• wawgn{p) for the AWGNC is 



wawgn{p) 



{Pl+P2 + ■■■ 



{pI+p'^ + ...+pI) 

The minimum pseudo-codeword weight of G denoted by 
""^mfrf ^^^"^^ minimum over all the non-zero pseudo- 

codewords of G. 

We now give definitions specific to the BSC. 

Definition 6: (Median for LP decoding over the BSC) The 
median noise vector (or simply the median) M (p) of a pseudo- 
codeword p distinct from the all-zero-codeword is a binary 
vector with support S = {ii,i2, ■ ■ ■ ,ie}, such that pi^, . . . ,pi^ 
are the e(= \{wbsc'{p) + 1) /2]) largest components of p. 

Note that for input y = M(p) for some non-zero pseudo- 
codeword p, we have cost(y, p) < and hence leads to a 
decoding failure (the output of the decoder, however, need not 
be the pseudo-codeword we start with). 

Definition 7: (Instanton for LP decoding over the BSC) 
The BSC instanton i is a binary vector with the following 
properties: (1) There exists a pseudo-codeword p such that 
cost(i, p) < cost(i, 0) = 0; (2) For any binary vector r such 
that supp(r) C supp(i), there exists no pseudo-codeword with 
cost(r, p) < 0. The size of an instanton is the cardinality of 
its support. 

An attractive feature of LP decoding over the BSC is 
that any input whose support contains an instanton leads to 
a decoding failure (which is not the case for Gallager A 



decoding over the BSC) ||34l . This important property is in 
fact used in searching for instantons. 

To summarize, evaluating FER vs SNR approximately re- 
duces to finding the set of most probable instantons and eval- 
uating their probabilities, multiplicities and, in the continuous 
case, also respective curvatures. Specifically, for LP decoding 
over the BSC and the Gallager algorithm, the slope of the FER 
curve in the error floor region is equal to the cardinality of 
the smallest size instanton (see (421 for a formal description). 
Understanding that the knowledge of the instantons allows 
efficient approximation of FER vs SNR dependence (which 
is our main task), we now discuss approaches to finding the 
set of instantons for a given error-correction setting in Section 

m 

IV. Searching for Instantons 

As explained above in Section |III1 instantons that control 
the large SNR asymptotic of the FER are the most probable 
noise configurations corresponding to decoder failures. Stated 
this way the problem of finding an instanton becomes an 
optimization problem, and all the remaining details of this 
section are related to efficient implementation of this, generally 
difficult, optimization problem. 

A. Instanton search for iterative decoding over continuous 
channels 

A straightforward optimization method for finding instan- 
tons in the case of a continuous channel is based on the 
standard (amoeba) optimization 143] and was discussed by 
Stepanov and Chertkov in 1401 . The main idea of the direct 
technique is as follows. One draws randomly a unit length 
configuration of the noise and finds a scale-up value which 
positions the re-scaled configuration of the noise exactly at 
the error-surface. Thus, incremental increase/decrease of the 
rescaling factor leads to decoding failure or recovery. Such a 
configuration and its probability are recorded, and this opera- 
tion is repeated {N—2) times, thus generating iV— 1 vertices of 
a simplex with respective probabilities assigned. Then, aiming 
to find a more probable point in the interior of the simplex, the 
current point is transformed according to the standard amoeba 
rules. The process is repeated until the size of the simplex 
becomes smaller than a preset accuracy, and the resulting 
most probable configuration outputs an instanton. The whole 
optimization is repeated multiple number of times, each time 
generating an instanton. The main advantage of the method 
is in its generality (it can be used for any continuous channel 
and any soft decoding algorithm). However, implementing this 
method is costly. Although one can use amoeba optimization 
method for LP decoding too, because of a certain property of 
LP decoding (it is easy to find an instanton (noise realization) 
corresponding to the output of the decoding which is a pseudo- 
codeword), the PCS method described in Section HV-CI is a lot 
more effective. 

The instanton-amoeba method easily finds the instantons 
for a code if the number of iterations in decoding is not 
large (less than 20). Increase of the number of iterations, nit, 
simply means longer computations. The other more important 



effect is associated with enhancement of irregular, stochastic 
component in decoding observed with n\t increase. One finds 
that already a slight variation in the noise can drastically 
change results. That makes the function that we have to 
optimize very irregular, which dramatically slows down the 
optimization process. 

In the case of large number of iterations in the decoder, 
with the check for a codeword in each iteration, it is not 
easy to come up with good starting point for the amoeba. The 
configurations from amoeba with small number of iterations 
(when the method is quite effective) are not very useful as the 
decoder eventually outputs a codeword. The following two 
ways to find such configurations were developed. Both are 
based on observations from numerical experiments. 

1) Input an instanton for LP decoder to the min-sum 
iterative decoder The instantons for LP decoding (as they have 
low weight) serve as good seeding noise configurations for 
amoeba, as they are in erroneous domain in noise space even 
for a decoder with a very large number of iterations. 

2) Limit the noise configuration on bits where the instanton 
for low number of iterations is supported. Work then with an 
optimization problem on these bits only, setting the noise value 
on all other bits to zero. In this way, the number of variables 
is much lower, so the optimization procedure is a lot easier 
to proceed with. The smaller is the dimension of the space 
in which the amoeba optimization is done, the easier is the 
problem. The instantons for low number of iterations usually 
have noise in a few bit locations. One can hope that if one 
increases the noise level on these selected bits (while keeping 
the noise at all other bits being exactly zero) then the noise 
configuration will "survive" a lot more iterations. The 12.45 
weight instanton (supported by 12 bits) for AWGNC and 410 
iterations decoder that is reported in POl was found this way. 

B. Instanton search for Gallager A/B decoders over the BSC 

In contrast to iterative decoding with continuous alphabet, 
the trapping sets and instantons for the Gallager A/B decoder 
can be found using certain combinatorial considerations which 
were first pointed out by Richardson llT6l and later investigated 
in detail in ||4T], (|44l, its), ED. The trapping sets for Gallager 
type decoders are closely related to trapping sets for the bit 
flipping decoders. 

C. Instanton search for LP decoding over the AWGNC 

For the LP decoding over the AWGNC, another sugges- 
tion for solving the difficult optimization problem faster was 
formulated in lf33l by Chertkov and Stepanov. This pseudo- 
codeword search (PCS) algorithm , originally stated for the 
continuous channel model, is based on the aforementioned 
relation between instantons and respective pseudo-codewords. 
Specifically, if a pseudo-codeword, p corresponding to an 
instanton, is known, then reconstructing the respective in- 
stanton p is equivalent to maximizing the probability of the 
noise under the condition that the probabilities of the noise 
configuration counted from the zero-codeword and from the 
pseudo-codeword, p, are identical, i.e. 

p= argmax„P(n)|p(„)^P(„^p). ^^q. (6) 



The idea of the method of |33l consists of throwing a 
sufficiently strong configuration of the noise (so that the 
resulting decoding is erroneous), decode it into a pseudo- 
codeword, and then assume that the pseudo-codeword shares 
an error-surface with the zero-codeword. Then the projected 
instanton is reconstructed using Eq. Q, even though the noise 
configuration, especially after the first iteration, is not an actual 
instanton. This procedure is repeated until the input and the 
output for an iteration give the same result. It was empirically 
shown in ll33l that such scheme formulated for the LP decoder 
outputs the sequence of noise configurations with probabilities 
monotonically increasing with the number of iterations and 
converging in a small number of iterations to an instanton. 

D. Instanton search for LP decoding over the BSC 

The PCS was extended to the case of LP decoding over the 
BSC by Chilappagari et al. in ll34l . The algorithm proposed in 
fi^ termed as the instanton search algorithm (ISA) is provably 
efficient and outputs an instanton in bounded number of steps. 
We summarize the algorithm below. 
ISA for LP Decoding over the BSC 

Initialization (1=0) step: Initialize to a binary input vector r 
containing sufficient number of flips so that the LP decoder 
decodes it into a pseudo-codeword different from the all-zero- 
codeword. Apply the LP decoder to r and denote the pseudo- 
codeword output of LP by p^. 

I > 1 step: Take the pseudo-codeword p' (output of the {I — 1) 
step) and calculate its median A/(p'). Apply the LP decoder 
to M(p') and denote the output by pa/, . Only two cases arise: 

• WBSciPM,) < WBScip'^)- Then p'+i = pm, becomes 
the l-th step output/(/ + 1) step input. 

• WBSciPMi) = WBscip'')- Let the support of A/(p') be 
S = {ii, . . . ,iki}- Let Sif = S\{it} for some if e S. 
Let r^j be a binary vector with support Si^. Apply the 
LP decoder to all r^^ and denote the it-output by p^^. 
If pij = 0,Vit, then A/(p') is the desired instanton and 
the algorithm halts. Else, p^^ ^ becomes the ^-th step 
output/(Z + 1) step input. 

The interested reader is referred to 1341 for a discussion of 
various issues that arise in the implementation of the ISA. 

V. Numerical Results 

This section summarizes statistics of instantons found for 
the [155, 64, 20] Tanner code 1361 performing over the BSC 
and the AWGNC and decoded by iterative and LP decoders. 
The Tanner code is a (3, 5) regular code whose Tanner graph 
has girth 8 ll36l . 

A. Instanton Statistics for the Tanner Code 

Gallager A algorithm: The most dominant trapping set in 
the error floor domain is the (5, 3) trapping set which has 
critical number 3. There are a total of 155 (5, 3) trapping sets 
each of which has an instanton of weight 3 |4l | (see Fig. |2(a)| i. 
There are 465 (4, 4) trapping sets each with critical number 
4. Hence, the slope of the FER curve in the error floor region 
is dominated by the (5, 3) trapping sets and it is equal to 3. 



(a) 



(b) 



(c) 



(d) 



Fig. 2. Illustration of the topological structure of instanton for different channels and decoders. Note that □ represents an even degree check node and I 
represents an odd degree check node. |(a)| A (5,3) trapping set with critical number 3 for Gallager A algorithm. Here o denotes a correct variable and • denotes 
a corrupt variable node, [(b)] An (8,2) trapping set for iterative decoding over the AWGNC. |(c)| The support of an instanton of size 5 for LP decoding over the 
BSC. 1(d)] The support of the lowest weight pseudo-codeword for LP decoding over the AWGNC. Note that the figure illustrates only the variable nodes with 
the lai'gest components. 



The trapping sets for the Gallager A algorithm are found by a 
combination of simulations and combinatorial considerations 
(see ][16), im for more details). 

Iterative BP: The instantons for 4 iterations decoder were 
analyzed by the instanton-amoeba method in 1251 . The 3 
lowest instantons were found, all of which contained a specific 
characteristic 12 bit structure. It turns out that this bit structure 
is what is responsible for errors even for very large number 
of iterations f40|. MC simulations show that the error floor 
asymptotic for the Tanner code under iterative decoder with 
large number of iterations is determined by these structures 
(resulting in effective distance of 12.45 (4^). All the trapping 
sets corresponding to the lowest weight instantons contain 
an (8,2) trapping set which is shown in Fig. 12(b)l For more 
details on the FER curves for different number of iterations 
and relevant discussion, the interested reader is referred to 
M- 

LP decoder over BSC: The ISA described in Section |IV| 
found 155 distinct instantons of size 5 (the corresponding 
pseudo-codewords have BSC weight 9). The support of each 
of these instantons is a (5, 3) trapping set shown in Fig. 12(c)l 
(from the symmetry of the Tanner code it can be verified that 
there are exactly 155 such structures present in the Tanner 
graph). The ISA also discovered higher weight instantons (see 
l34l for more details), but the instantons of size 5 are the most 
dominant ones in the error floor region. 

LP decoder over AWGNC: The PCS algorithm of 1331 
found many low-weight pseudo-codewords (16.4037 being the 
least weight pseudo-codeword as found by the PCS). The 
weighted-median noise configurations (instantons) (see l33l ) 
corresponding to various low-weight pseudo-codewords have 
high noise at 5 variable nodes corresponding to the (5, 3) 
trapping sets. In fact, the respective BSC weight 9 pseudo- 
codewords have low weight on the AWGNC also (but not the 
absolute lowest!). The support of each of the lowest-weight 
pseudo-codewords is large but the components in the variable 
nodes corresponding to the (5,3) trapping set have maximum 
value (illustrated in Fig. 12(d)] ). 

An important insight gained from this comparison is that the 
decoding failures for various algorithms on different channels 
are closely related and are dependent on only a few topological 
structures. These relations can be exploited to find instantons 



for a given decoder on a given channel based on the knowledge 
of instantons for another already analyzed decoder, which can 
even be performing over another channel. This relation is also 
suggestive for design of a better code, the idea substantiated 
in the next subsection. 

B. Code Design for Increasing the Smallest Instanton Size 

In l26l . l45l . it was shown that the minimum pseudo- 
codeword weight (for LP decoding) and the minimum critical 
number (for Gallager A/B decoding) of a code increase with 
the increase in the girth of the underlying Tanner graph. 
While girth optimized codes are known to perform better 
in general, the code length and degree distribution place 
a fundamental restriction on the best achievable girth. Ob- 
serving that the instantons for different decoding algorithms 
performing over different channels have a common underlying 
topological structure (e.g. the (5, 3) trapping set in the case 
of the [155, 64, 20] code), it is natural to discuss design of 
a similar but new code which excludes these troublesome 
structures. In fact, this suggests a natural code optimization 
technique with an improved instanton distribution. Starting 
with a reasonably good code (constructed either algebraically 
or by the progressive edge growth (PEG) method l47l ), 
we find the most damaging instantons and their underlying 
topological structure. We then construct a new code avoiding 
such subgraphs (either by swapping edges, by increasing code 
length, or utilizing a combination of both). We iterate this 
procedure till the code can no longer be optimized or reaching 
a computationally unbearable complexity. 

For the Gallager A decoding, it has been proved in l48l that 
codes with Tanner graphs of girth 8 which avoid the (5, 3) 
trapping set and weight 8 codewords can correct all the error 
patterns of weight 3 or less. While proving a similar result 
might be difficult for the iterative decoder over the AWGNC 
and the LP decoder, such considerations nonetheless play a 
role in our code design strategy. An algorithm, suggesting 
construction of a code meeting the Gallager A-related con- 
ditions, was provided in l48l . This algorithm can be seen 
as a generalization of the PEG algorithm l47l . Given a list 
of forbidden subgraphs, at every step of the algorithm, an 
edge is established such that the resulting graph at that stage 
does not consist of any of the forbidden subgraphs. (The PEG 
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Fig. 5. Pseudo-codeword weight distribution for tlie Tanner code and the 
new code on the AWGNC as found by running the PCS algorithm 2000 times. 



Fig. 3. Compaiison of the PER performance of the Tanner code and the new 
code under the Gallager A algorithm. Plotted also is the asymptotic prediction 
made using the statistics of lowest weight instantons. 
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Fig. 4. Instanton weight distribution for the Tanner code and the new code 
for LP decoding over the BSC as found by running the ISA 2000 times. 



algorithm is a special case forbidding cycles shorter than a 
given threshold.) 

Using the algorithm proposed in 1481 . we constructed a new 
code of length 155 with uniform left degree 3 and with most 
check nodes with degree 5. By construction, this code avoids 
(5, 3) trapping sets. This results in a steeper FER slope of 
4 in the error floor domain under the Gallager A decoder, 
as shown in Fig. [3] The dominant trapping set for the new 
code with critical number 4 is the (4, 4) trapping set (an eight 
cycle) and has multiplicity 662. Fig.[3]also shows the predicted 
performance at very low e (the method to predict the error 
floor performance using the trapping set statistics is described 
in detail in pTl). 

The minimum weight instanton for LP decoding of the 
new code over the BSC found by the ISA is 6 (we have 
independently verified that the code is in fact capable of 
correcting error patterns with up to five errors by exhaustive 
search). Table |T] shows the instanton statistics for the Tanner 
code and the new code found by running the ISA with 20 
random flips and 2000 times. The statistics of number of 
unique instantons for the two codes as a histogram is illustrated 
in Fig. |4] The pseudo-codeword weight distribution for LP 
decoding over the AWGNC for the two codes is shown in 
Fig. |5] The FER performance of the Tanner code and the new 
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Fig. 6. Comparison of the FER performance of the Tanner code and the 
new code under LP decoding over the BSC. Plotted also is the asymptotic 
prediction using instanton statistics for the Tanner code. For the new code, 
as the total number of instantons of weight six is unknown, different curves 
are plotted (labeled 1,2,3) assuming different (200,2000,5000 respectively) 
number of instantons of weight six. 



code under the LP decoder over the BSC is shown in Fig. 
|6] While all the lowest-weight instantons for the Tanner code 
have been found by the ISA, the same cannot be said of for 
the new code 0. Hence, we can only predict the slope of the 
FER curve in the error floor region and not the exact value. 
This can be remedied by running more trials of the ISA or by 
studying the automorphism group of the code and exploiting 
the structure of the code to find the multiplicity of the lowest- 
weight instantons. In Fig. |6] we have plotted the predicted 
FER curve assuming different values (200, 2000 and 5000 
respectively) for the number of instantons of weight six. All 
the above statistics illustrate the superiority of the new code. 

^The standard way to find out whether our instanton search exhausted all 
the unique configurations is as follows. Assume that there are A'^ unique 
instantons of a given weight and in each trial ISA finds all of them with equal 
probability. To estimate the number of ISA runs required for finding all the N 
instantons, one notice that if — 1 instantons are already found the number 
of trails required to find to the last instanton is ^ N. If all but two instantons 
are already found the number of ISA trials required is N/2. Therefore, the 
average number of ISA trials required to find all the instantons is estimated as 

N+N/2+N/3+- ■ ■ N/{N-l) + l = Ar(l+l/2+l/3H hl/N) turning 

to A^lnAf at Af ^ oo, i.e. N\nN trials ISA reliably finds N instantons. 



TABLE I 

INSTANTON STATISTICS OBTAINED BY RUNNING THE ISA WITH 20 RANDOM FLIPS AND 2000 INITIATIONS FOR THE TANNER CODE AND THE NEW CODE. 



Code 


Number of instantons of weight 
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10 


11 


12 


13 


Tanner code 


Total 


715 


194 


248 


230 


295 


201 


74 


10 


1 


Unique 


155 


177 


238 


228 


295 


201 


74 


10 


1 


New code 


Total 




106 


409 


622 


508 


247 


62 


11 


1 


Unique 




80 


397 


617 


508 


247 


62 


11 


1 



VI. Conclusion 

In this paper, we presented a comprehensive description 
of various instanton based techniques for the analysis and 
reduction of error floors of LDPC codes. The most powerful 
method discussed is the pseudo-codeword/instanton search 
algorithm, designed specifically for the LP decoder Using the 
instanton-based technique for analysis of sample (intermediate 
size) codes, e.g. [155, 64, 20] Tanner code, we conclude that 
the underlying topological structures of the most probable 
instanton, found for the same code but different channels 
and decoders, are related to each other Understanding of 
the graphical structure of the instanton and its relation to 
the decoding failures leads to a method to construct codes 
whose Tanner graphs are free of these structures. The instanton 
technique, applied to this code and also complemented by the 
direct Monte Carlo simulations, confirm the success of the 
new code improvement strategy. 

Future work includes: (1) refining the above techniques 
and applying them to longer codes, (2) developing improved 
semi-analytical methods for FER estimation, more specifically 
combining instantons and MC in order to obtain a good 
approximation of the entire FER curve, (3) optimization of 
decoders to reduce error floors, and (4) finding other combi- 
natorial strategies for designing low error floor codes, such as 
judicious removal of the lines in a finite geometry leading to 
point-line incidence matrix free of trapping sets. 
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